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SUMMARY 



ProMem 

The problem was to develop candidate' pilot performance measures for five undirgraiiuate pilot 
training (UPT) contact maneuvers flown, in the T-37B aircraft Oazy 8, banel roll, split S, cloverfeaf, and 
landmg). Existing methods of pilot performance measurement are largely subjective . have questionnable 
validity and reliability, and are relatively insensitive to changes in performance except for gross variances. 
This makes existing methods less than pptimal for training applications per se, and of little scientific value 
for training research (e.g., flight simulator validation studies) where transfer of training must be assessed. 
This effort was designed to develop improved measxwement methods which overcame many of the 
disadvantages inherent in existing techniques. The methods were intended for evaluatioriVcro8s-\c^datlqn, 
,and application in the advanced simulator for undergraduate pilot training (ASUPT). \ 

Approach > ' 



The approach was one of two alternatives punued in parallel for the above application. It is 
characterized by researcher development of candidate measures and assurance of their content validity, 
automated computation of the measures and execution of various empirical vahdation tested 
(The alternative method (ConneUy et al., 1974) involves computer generation of candidate measures from a 
postuUted inclusive family thereof, and execution of empirical validation tests, foUowed by researcher 
analysis of results and assurance of the.measures' content validity..) The first step of the approach was to 
uialyze eaCh maneuver using a hybrid venion of ftmction and task ailalyses specifically tailored io 
identification of candidate measures. Then several, types of measures were defined which, cdlectively 
support performance assessment over aU maneuver segments. Next, specific mea^remen>fonnulae were 
developed for each segment in accordance with the results of the analysis. This ipdudeithe development of 
kltemative techniques for combining measures over segments and maneuver^. ^inaUy, software was 
developed to compute measures, from data recorded on a T-STB aircraft, and execute empirical validation 
tests. , - y . 

Results 

Each maneuver was analyzed and sectored into ftmction and task segments. The ftmction segnvsnts 
Identify portions of the maneuvel: wherein the set of dominant measurement variables is consistent This 
Identifies sections of each maneuver in which the piJot^ primary control fimctions involve consistent 
n)sa«i;rable variables, and dictates the typfes of measures (continuous and discrete) that are appUcable to 
_^w^Kcontrol performance. The task segments identify portions of each ftmction segment wherein the 
relattonsftiK among dominant measurement va.riables are consistent'^ This identifies sections of each 
maneuver ilMihMiUif pilots' primary controNftincti^nrlhemselves remain consistent, and suggests the 
g)eciflc natiire ormeasures that are appUcable^rperformance assessment within the respective task 
segments. \ / ^ ' ' . * 

Several tyWof m^astues were identified which satisfy measurement requirements over all task 
segments, based o^appifcation of the above analysis to five maneuvers. Specific measurement formulae 
were then developelfor each se^nt, exploiting the measure-types previously identified. Software was 
developed and unplAnented orf a Sigma 5 computer for accomplishing the following tasks: (1) read and 
remove noise from/nw aircraft data; (2) produce print outs and plots of raw and smoothed data at 
user-spexafiable saAplbg rates; (3) automatically detect boundariesjjftask segments; (4) compute criterion 
(reterence) &in^(ns from skilled performers' data, employing user-specifiable dependent and independent 
variables; (5)4mlpute tadc-segment measures of 6 types, parameters of which are changeable by the user- 
compute segment and maneuver summary measures using ^temative methods under study; (T) print 
results of maneuver segmentation, criterion ftmction generation, and measure computations; and (8) 
pertom and pnnt results of several empiri(^ validation tests. . - ' . ^ 

I ' - • ... 

Conclusions ' . ' ' 

■ ' A unique and relatively effective technique has been developed and appUed for identifying candidate 
pertormance measures for continuous operator cqjftrol tasks. The specific objective of' developing such 

i , . 
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measures for five T.37 pflot maneuvers has been accomplished. In view of the analysis ^d derivation 
techniques employed, the candidate T-37 measure^possess content validity and arc fairly compfehensive. 

The segmentation of the maneuvers and development of methods for automatically detecting the 
icgment boundaiict provides an impoyttnt and necessary adjunct to tn envisioned measurement system. It is 
not an especially novel.idea, but its criticality has been tmder emphasized in the i^ast, and it has not been 
previously developed and applied to the extent accomplished in this study. Two of the maneuvers examined 
(cloverieat and approach and landing) are characterized by more than one continuous function segment, 
.witjiin which are numerous ditfering task segments, eadi dictating the application of um'que measures The 
-o^r three maneuvere, are larg^y characterized by a sin^e continuous ftmction scgmept. However, there 
are still differing task segments within each function segment, and the initial portions of all maneuvers are 
characterized by discrete funAion segments that differ grosdy in measurement requirefnents from their 
continuous counterparts. In view of this type of analysis ds ap|died to the five maneuvers, it is easy to see 
why attempts to apply a single continuous measure over an entire control task (or over inappropriate 
portions thereoO might expectedly result in enoneous deductions about the invalidity of the (valid) 
measl\ire. 

Finally', the' software that has been developed and. implemented is unicjue in both capabl^ty and 
flexibility. Inputs to the programs consist of raw performance data and .user-specified measures for the 
various task segments of each maneuver. Outputs consist of the values of computed measures, results of 
several validation tests that arc appropriately 'computer-implemented, and several summary measures 
computed. using alternative techniques under study. The user can interact with the software effectively in 
pursuit of an analytic, iterative approach to the development and validatioa of operator performance 
measures. * ' 

' • The griginal objectivcss of this program hiduded its extension through and includiag a 
criterion-related validation phasc^ It is unfortunate that these objectives could not be fully realized due to 
non-technical problems that;uiterfercd with, and ukimately prevented, the collc^ction of required T-37 data 
and completion of this phase of the program. Despite this, it is hoped that the work reported here wiU 
benefit other researchers investigatiiig similar areas by providing some additional tools and ideas and 
demonstrating their application, at least in part, to an exemplary measurement problcjn. 
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CANDIDATE T-37 PILOT ^ERFORlfiNCE MEASURES - 
^/ FOR FIVE CONTACT MANEUVERS 

I. INTRODUCTION 

I The purpose of this study was to develop candidate T-37 pilot performance measure^ for. five 
maneuvers taught in the Air Force undergraduate pilot training (UPT) program. In addition, the'^study 
included development of software for computing the candidate measures from performance data recorded 
on a T-37B aircraft, and performing validation tests. This report documents the approach, the maneliver 
analyses, the derived candidate measures, and an overview of the implemented computational techniques.** 

Existing methods of pilot performance measurement are largely subjective and hi(ve .questionable " 
validity and reliability. This.makes them less, than optimal for training applications in gerferal, and of little 
scientific value for simulator training"" research, where transfer of training must be assesse^. This effort was 
designed to ievelop improved measurement methods which overcome many of the disadvantages inherent 
in exiisting techniques. The specific goal of developing objective measures of T-37 pilot performance 
emanated from a requirement for a comprehensive simulator training reasearch capability for effective use 
of the advanced simulator for undergraduate pilot training (ASUPT). ^ 

The research effort, as originally designed, included the development of candidate measures based on 
their content validity, and the conduct of criterion-related validation studies using student data to be 
recorded on a T-37 aircraft. Nontechnical problems prevented the required data collection, however, and 
the study had to be confined to the development of candidate measures, as reported here. 



^ '0, APPROACH , 

There are twp basic alternative Approaches to deriving candidate measures for a given operator 
* performance task. The approaches differ primarily in. (1) the order in which various types of 
measure-validity are asjsured andy6r tested, and (2) the loads place J onnnan versus computer as a result of 
research task aUocatiorjs. ^ ^ . • 

Validation tests Jean be grouped into two fundamental classes, the major distinction being whether or 
not they are criterion-related. It is also useful, from an operational standpoint, to consider validation tests 
from the view point of whether or not they can be conducted using a digital computer. For example, it is 
, extremely expedient to automate tests of concurrent yalidity, where* candidate measures are examined on a 
'comparative basis with other independently denved measures of performance. On the other hand, it would 
be quite difficult to automate tests of content validity, where the aspects of performance^ assessed by the 
candidate measures are compared with the behavioral objectives underlying the perfoonance tasks, as 
c^fld^enced (usually).by task- analyses and stated performance ol^qtives. ' * ' 

The, research jobs associated with deriving candidate measures include. (1) selecting the specific^ 
measures to be explored, (2) assuring their content validity, and (3) testing them empirically for other types' 
of validity (e.g., concurrent^. The first job can be performed by^man or computer. The second is most 
efficiently performed by man, since it requires cognitive proce^ing of diverse (often unquantifiable) 
information. The third is mosf • efficie^ntly performed by a computer, since it requires standardized 
processing of large quantities of dgta. 

The two approaches in^ntipned in the opening paragraph are, respectively, to. (1) assign the computer 
the jobs of generating^feasiole Candidate measures and performing empirical validation tests, followed by 
manual analysis of results and assurance of the measures* content validity, or (2) assign man the jobs of 
selecting feasible candidate measures and assuring their content Validity, followed by computer tests of the 
measures for various types of empirical validity. The first approach (Connelly, Bourne, LoentaU& Knoop, 
1974) places the greatest research load on the computer, and has the ad^ntages of assuring examination of 
a broad spectrum of fneasures and of being universally applicable across diverse performance tasks. It is 
based on a relatively new and unexploited technique of measurement research, however, and is in an 
exploratory stage of development. The second^pproach places the greatest research load on the man, and 
has the major advantages of traditionality and'^apparent simplicity. It *is subject to limitations primarily on 
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the number and types of measures (from the set ofj^ valid measures) that can be identified and explored, 
as constrained by the ingenuity and availabte time of the researcher and the number of measurement 
problems lo be addressed. \ ^ * ^ . 

This latter approach is the topic of the present report*. It was pursued m parallel with the alternative 
approach for developing measures of performance on 5 UPT maneuvers^, lazy 8, barrel roll, spht S, api^roach 
and landing, and cloverleaf. • . * 

*- « 

in. ANALYSIS OF UPt MA?reUVERS 

0' • 

General *• . - 

This section documents the rationale and methods used for the analysis of the five UPT trainmg 
maneuvers. The maneuver analysis includes segmentation of maneuvers, identificatioji of pilot skill^foL 
each segment, and identification of candidate measures of performance for each segment. While the metho3 
was develop for analysis of five spedfiq maneuvers, it is a general one that can be applied to other 
operator performance task5. ^ . ' • 

In addition to a description of the methodology, a list of definitions of critical terms is presented. 
These terms, ai^ irj common use in human engineering and many other fields but may have different 
meanings de^feh^g on one's indoctrination. They arc defined to reduce ambiguity. The terminology is 
generally derive from (Meister, 1971, Miller, 1955, Morgan, 1963, Woodsen & Conover, 1976). 

Definitions of Critical Terms ' ' - • 

A task is one or more activities performed by a sin^c human operator to accomplish a speafTed 

objective. ^ - • 

* . • 

A Jiarnere ras^ is an afl-or nothing task iri response to a given signal. * 

A continuous task is con tinuotisly 'performed activity based upon feedback. 

A critical discrimination is a perceptual process .tp ascertain a particular variable value to within a 
specified tolerance. An entire maneuver can be consiBe'rcd as a set of tasks bounded by critical 
discriminations. These discriminations form the base knowledge for a decision to continue, tcjfimtiate an 
error compensation, or to abort a maneuver. 

A control function h one ^of the operational requirement^ necessary to achieve a portion of the 
man-maphinc total Qutpyt, For example, one control fanction could be 'control of pitch necessary to 
achieve a dimbing turn in a Lazy 8 maneuver. ^ 

The set of-dominant control functions are ihose control functions without which it is impossible to 
achieve the intended portion of the maneuver. • ' * ^ 

Subordinate control functions are those used to assisl in the effiaent^rforraancc of a maneuver but 
which ^ not essential. . , * ^ , ^ " 

Reference variables are those aircraft fjight variables which are used to define the criterion ;nan?uver s 
trajectory. . • » 

Dominant measureriient variables are those flight variables which are highly correlated with the 
control fti^ctions which are important to proficiency measurement. - ♦ 

A function segment is that portion of a maneuver in which the set of dominant measurement 
variables is consistent. These segments are classified in two groufDs, locus segments in which the dommant 
measurement variables are continuous/ and sequence segments in widch the dominant measurement 
vaiiabl^ relate discrete events to a continuously changing variable. 

A task segment js that portion of a function segment in which the relationship among dominant 
measurement variables is consistent. A task segment is bounded by critical discrimmations, Task^segments 
arc the basic measurement segments of that maneuver. 

A skm is the ability io use kriowledgc to perform n^anual operations in the achievement of a specific 
task objective In a manrier which prc^vides for the elimination of irrelevant action and erroneous response, 
litis conccptuaiizition exists only in conjunction with an individual task and is refieaed m the quality with, 
which this task is performed. . ' " 



An event is an activity within the total event sequence. * * / , 

An event sequence is a connected series of specified activities occurring in time or in some other 
continuously changing variable. , / 

Methodology 

Methodology Rationale. The objective iii the selection of an analytical 'methodology is the 
development of pilot proficiency metres. The process used to denve candidate pilot proficiency measures 
is a hyijrid version of a function andtask analysis. Normally, a function analysis is used to consider vanous 
configurations sihd allotments of functions between the man and the machme with the objectivfrbemg the 
selection of an optimum design. A task analysis is used to investigate performed tad;s m order to ascertain 
those actions which should be appropriated to the equipment and those to be assigned to the human 
operator in order to achieve overall system efficiency. Smcc a system design or function allotment is not 
required here, the function and task analysis used can be simplified. 

One basis for the systematic analysis of UPT maneuvers m order to establish pilot proficiency 
measures is the categorization of human error. In Meister (1971)^ human error is categorized into three 
types: 

1 . Failure to perform a required action; that is, an error of omision. 

2. Perforrmnce of the rnission in an incorrect manner; that is, an error of conurussion. ^ 

3. Performance out of a sequence or at an incorrect time. 

Function and task analysis' cari be used to ascertam these classes of errors, their c^ses, and their 
cause/effect relationships. However in training and evaluation, one caiinot be constrained by limitmg 
definitions of human error and by analytical approaches tailored to eqnjpment«<iesign and devctppment. 
Recognizing the potentiality of such took, accepting their limitations, ^and usmg such kn(^wledg^'to develop 
new tools provides an efTective anaiytica! approach. This has accomplished ^Hf^^ .analytical 
methodology described here. * \ . 

Historically, maneuvers were developed to train student pilots for combat so that when the aircraft 
man-maohine system is pushed to its capabilities, the pilot has an adequate recovery reaction, denved from 
a matrix of possible alternative^. The learning task for the neophyte pilot is one of acquiring new maneuvej^ 
res^K)nses The advanced pilot learning task H ia the selection and utilization of these response^ m^^^ch a 
way as to fulfill a mission objective. Combat skilU require a high proficiency in response selection^as \^ell as 
response (performandc). Therefore, pilpt skill evaluation tools require teclmiques which allow gathenngof. 
information in both areas. ^ > ' j 

*' * 

In each of the five UPT maneuvers qSnsidered he^, the importance of task selectivity is magmfied 
during pilot generated error compens^on tasks. One approach used in performance measurement is 
assessment of a simple difference betweea the achieved path ^d the reference path. Thus, when a pilot 
commits a flight control error this measure continues to penalize him ioitil he recoverS.^ Tliis ism spue of 
the fact that he fnay be using coffcct recovery techniques. Such an approach eUminates measuje seiisitivity 
to the deals involved in the recover tasks performed by the pilot* The fireedom of response selection m ' 
such tasks is greatly increased over the reference maneuver task, indicating a requirement to develop a high 
skin level in selex:tion of error recoveries. Therefore, the .analytical tools required for this maneuver analysis 
should include an evaluation of pilot recovery tasks. 

Steps Employed in the Developfnent of Pilot Profidency Measurements. The approach starts with the 
i^enng (data collection) and study of basic background information including, but not limited to,SirCTaft 
dynamics and maneuver data. Primary data sources induded maneuver analyses (Baum, Smith, & Goebel, 
-1973) preparcd with Hit assist^ce and aptive participation of Hq Air Tralnmg Command {ATC) officials 
andamcnt T-37B instructor pflots. Supplementing these basic analyses were ATC flight manuals and 
tecnmcal orders for the T-STB, kn'd maneuver training films used in the academic portion of UPT (see bibho- 
graphy, -^ Finally, plots" of real data recorded on a T-37B aircraft during performance of the manemiers 
were used to obtain realistic estimates "of ranges, tofcrrances, and critena in instances where other data 
Sources produced conflicting? or otherwise ambiguous information. (The. plots were produced during a 
^qt\\^ ^"^y establish the feasibility' of automated T.37 performaAcc measurcmem (Knaop & Weld^, 



The mtial ;evifw of th^.datVp^ovided the in vesti^ting team an .excellent general understanding of 
the performance objf gaves of each maneuver. In-depth revfews of the data, duringjjie «ubiejquent work on 
function and task segmentation and identification of candidate measures, assured that maxinmm use was 
made of all existmg information and included cross-checks between the vj^rious data sources. ^ 

The, following five sieps(/unefton and task segmentation) sue performed uniquely for each maneuver 
and are implemented m Section VII. The first step is to identify the pilot pe^rformance tasks required for 
each maneuver. This provides a sequential list of all the tasks including oentrol, perceptual, and other 
required for the performance of the maneuvers. 

The second step is to 4evelop tm^e charts for the delineated maneuver tasks for each maneuver. Such 
a chart indicates the appropnate time domain for each 'task, the sequence of occunencc, and the 
appropnate overlappmg of t^ domains. The chart includes control tasks, perceptual tasks, and critical 
discriminations. . \ ' 

The^ third, step is to determine the' control functipns required for each task of each maneuver. 
Emphasis is placed on identification of doiiunant control functions. The determinationof ui?sr3ominant 
control functions necessitates the observation of their tirne relatjojiships. This directly l^adsinto step four 
which is to identify the nianeuver ftinction segments.,. ' 

Step five is to establish the task segmerrts. This segnwntation process follows the function 
segmentaUon duetto the prerequisite of estabUshmg the appropriate operational requirements for the ^ 
accomplishment of these^spedfic tasks. * - . 

The final three steps (proficiency measure development) are iinplemented in Sections VII and Vlil of 
this report and are as follows: 

1. Establish flight error measurement rationale, ' 

2. Develop candidate.proficiency measures for-cach tadc segment, and ^ 

3. Develop candidate methods of combining proficiency measures. ^ 

"rV. FUNCTION AW TASK SEGMEOTATION^OFm " 

General . . . 

* 

The analysis techniques discussed m the previous sections ^ere applied lo the five UPT nianeuvers to 
identify the.measure vanables and measurement rationale required to specif) candidate pilot proficiency- 
measures. The procedure employed was as follows. (1) Identify pilot performed tasks for eabh maneuver, 
(2) Develop time charts for the pilot tasks, (3) Determine control functions required for each task, (4) 
Identify function segments, and (5), Establish task segments. Ip^c employed in the methodology for the 
selection of the measurement vanables is shown in Figure 1. For each tadc in a maneuver, the associated 
^ntrol functions are matched against the measurable fli^t variables to determine which of these variables 
are highly, correlated with those confrol functions. The result of this tomparison is the identification of 
candidate measure vanahfes. The najufe of the dominant contr6l functions and associated measure variables 
IS examined so that candidate measure operations and methods of combining measures can be devised. 

Lazy 8 * - * . 

* Table4 lists the vanous tasks which must be compljeted m order to perform a lazy 8 maneuver. These 
tasks are organized into appropnate function segments and task segments. The first function segrftent is the 
"mitiaT* segrnent m which the establishment of the prcdnaneuvei fli^t .attitude is attained. There, are 
contmuous tasks mvoWed m these mitial function segments, however, this does not warrant the treatment 
of these mitial segments as locus segrnwUs due to the nature of their discrete outputs. In lieu of this, th^ 
initial segment is treated as a sequjencie se^nent which rtlateS^events in time. As an example of a continuous 
task being treated as a discrete event, conader the attainment'of an airspeed of 215 knots. To do such, ^ 
pilot must consistently monitor his airspeed mdicatoi,apj!ppiiatel> adjust the throttle, and either wait for 
the •thrust to boost the air;?pecd to the desired level or pitch down in order to attain the requircd^vclocity. 
'In such a task the primary importance is on the attainment of the. required airspeed and less importance Is 
placed on the method of airspeed attainment. ' ' ' ' ^ * 
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Table L LazySjTasto ^ 

Functton StgmtYit inHUl 



Task Scgmentr Initial \\ 

1. Establish level flight at 8,000 feet. . 

2. Set power at 90% RPM. 

3. Attain an airspeed of 21 5 knots. * • 

4. Visually clear the area^* » . , 

5. Select a horizon reference point directly off the right wing tip. 



Function Stgmtnt-Lw S 

Task Segment: First Eighth . ^ 

6. With the aucraft nose projection on the horizon, form a symmetric ei^t laying on its side with the . 
horizon as the long axis. 

7. Blend aileron, rudder and elevator pressures starting a gradual climbing right turn, continuing to 
monitor and control the pitch, bank and turn rate so as to attain a maximum pitch attitude with 45^ 
of bank at 45'' of turn. 

8. Monitoj and control the turning rate so as to attain the reference point in the center of.the 
windscreen at 9(f of tum. ' • ' 

9. Monitor and control the pitch and turning rates so as to attain a 100 knot maximum lorss in airspeed 
at 90'' of tum. ' - ^ 

10. Monitor and control the roll so ai to attain a 90** bank at(4he lowest velocity point. 

1 1 . Establish and discem the maximum pitch attitude with 4^** of bank when the aircraft has turned 45**. 

Task Segment: Second Eighth 

12. Lower the nose slowly to the hojizon and toward the reference point while increasiiig the bjmk, 
continuing to monitor and control tiie pitch,l)ank, and tum rate so as to attain a 90® roll, zero pitch 
and 1 1 5 knot velocity when the aircraft has^tumed 90*. 

13. Cross and discem the horizon with a 90** bank with a 100 knot loss in entry airspeed with the 
reference point in the center of the windscreen at maximum barometric altitude. 

Task Segment: Third Eighth • ' ^ 

14. Decrease the bank at the same rate as it increased m order to describe the same size curye below the 
horizon as it did above while continuing to'monitorand control pitch, b^nk and tum rate ^ as to 

f .first attain a'minimum pitch with 45**^of bank when the aircraft has turned 135*. 

15. Monitor and control the turning rate so as to attain the reference point off the left wing tip at 180* 
i of tum. 

16. \ Monitor and control the pitdi ^d turning rate so as to attain 215 knots airspeed at 180* of tum. 
, 17. Monitor and control the roD so as to attmn level flight at thefKghest velocity point. 

18. Establish and discem the minimum pitch attitude with 45* of bank when the aircraft has turned 
135*. 

Tadc Segment: FourthEighdi " 

Vi, Raise the nose slowjy to the horizon continuing the rdl out so as to attain level flight at 8,000 feet 

altitude and ^tS.k^ots ai|$peed when the aircraft hSs turned 180*. .. 
20. Establish said drscfai the position* of the reference point directly off the left-wing tip while at 8,000 

feet and 215 knots.and at the end of the 180* tum. 

Task Segmcnt:-Fifth Eighth 

2L Immediately begin and control a climbing turn in the direction of the reference point at the same 
rate, same proportion, but in an opposite rotational direction as in the first tum, colitinuing to 
monitor and control the pitch, bank* and turn rate so as to attain a maximum pitch attitude with 45* 
of bank at 45*" of ttim. ^ ' ' - ' ^ 

17 . . . . 

14 ^ 



Table 1 (Continued) 



Function S«9n>«nt U^xy I 

' ' 7 

t2. Monitor and 5X)ntrol the turning rate so as to attain the reference point ih the center of the 
windscreen at 90° of turn. ^ 

23. Monitor and control the pitch and turning rates so as to attain a 100 knot maximum loss in airspeed 
atM^'oftum. 

24. Monitor and control the jolTso as to attain^a 90^ bank at the lowest velocity point. 

25. Establish and discern the maximum pitch with 45° of bank when the aircraft has tumed 45°. 

Task Segment: Sixth K^th 

26. Lower the nose slowly to the horizon and toward the reference point while increasing the bank, 
continuing to monitor and control the pitch, bank and tum rate so as to attain a 90° roll, zero pitch, 
and 115 knots velocity when the aircraft has tumed 90°. 

27. Cross and discern the horizon with a 90° bank with a 100 knot loss in entry airspeed with the 
^ ^ reference point in the center of the windscreen at maximum altitude. 

Task Segment: Seventh Eighth 

28. £)ecrease the bank at the same rate as it mcreased in ^der to descnbe the same size loop below the 
horizon as it did above while continuing to monitor and control pitch, bank and tum rate so as to 
•first attain a minimum pi^ch with 45° of bank when the an^ft has tumed 135°. 

29. Monitor and control the tuming rate so as to attain the reference point off the ri^t \ying tip at 180° 
of tum. ■ ' 

30. Monitor and control the pitch andiuming rate so as to attain 215 knots airspeed at 180° oftum. 

3 1 . Monitor and control the roll so as to attain level flight at the highest velocity point.^ 

32. Establish and discern the mimmum pitch attitude with ^45° of bank wheo the aircraft las tumed 
135°. - * , • ' 

Task Segment: Eighth Eighth 

33. Raise the nose slowly tb the horizon, continuing the roll out so as to attain level fBght at 8,000 feet 
altitude and 215 knots airspee4 when the aircraft has tumed 180°. - , . 

34. Estabbsh and discem the position of the reference point directly ^ff the right ,wing tip while at 8,000 
feet and 216 knots and 180° tum. 



Tasks 4 and 5 of the Lazy 8 initial segment are pejceptual tasks and cannot directly be measured from 
the aircraft fHght variables. 

The next and dominant function segment of the Lazy 8 maneuver is called the Lazy 8 segment. A 
consistent set of control functions is dominanf throughout the entire J-azy 8 function segment, however, 
there are eight task segments contained within the Lazy 8 function segment. Each task segment ends with 
specific ciitical disciimination^ of flight variable values. Task 6 is the continuoifs perception task of fprming 
a horizon projection of an 8 laying on its side. This visualization qan be closely approximated by, a plot in a 
pitch ^nd roD space. Tasks 11, 13, 18, 20, 25, 27,32 and 34 are all critical discrimination tasks marking the 
ending of a prior task segment. ' - «• 

AH of the tasks listed in Table 1 are plotted in their associated time domains in the Lazy 8 time chart 
(Figure 2). In this time chart, the appropriate task segments which follow sequentially in time are listed af 
the top. All continuous feedback t^s are indicated by circles with lines extending until their termination 
point in time. The critical discriminations are drawn as triangles. The tasks have been broken down to their 
smallest components' (segments) which are clearly discemed by the appropriate placements of the catical 
discriminations. (A higher level of task segmentation is possible. For example, there are four equivalent 
domains in this task time sequence reflecting similar ta$ks which could liave been used for task 
segnientation. Ffowcver, tlie smallest task components arc use4^for thi|^study.) 

With Ihc exception of the initiaT segment the Lazy 8 maneuver is dominate4 by roll and pitch. 
Therefore, the emplo/ed measurement variables must directly rcjlect these two domlnMjt control 



functions. Other control functions which influence the system's output are yaw, trim, RPM, the rate of 
descent 'and climb, heading, altitude, and airspeed. All of these are factors which the pilot must adequately 
control. -However, roll and pitch maintainjlieirjloniinancc throughout the entire Lazy 8 function segment. 

Figure 3 indicates four fli^t variables (puchjs^itude, airspeed and roll) along with identification of 
task segmentation and function, segmentation. NumB^is^hown in parentheses adjacent to task segment 
numbers indicate similar (except for roll) task segments ebewhere. in the maneuver. For exaniple. Task 
Segment 1 is similar to Task Segment 5 except that 5 is accompHdied with a negative roll aijgle. Inspection of 
the pitch and roll plots show that the critical discrimination tasks are those' components where the values of 
pitch and roll attain maxima, minima, and zero. 

Approach and Lancjing 

Table 2 lists, the taSks involved in a left-tum landing pattern maneuver. All^lums associated with th^ 
holding pat'tcm, including the turn into the initial, can be considered as discrete events leading up to the 
turn entry over the pitch out point. Tasks 5, 8, and 9 are all perception tasks which ^e ;iot measttred 
directly. - ... 

The usual operational method is to. evaluate performance of the maneuver yp to touchdown* This 
reflects a training policy which allows a greater number of landing attempts per unit tSne by landing to 
touchdown and then immediately taking off C*Touch and go^O- However for maneuver analysis the 
touchdown, the nosedown, and the rollout tasks are considered as part of ^ landing maneuver. \" 

Figure 4 is the time chart for the normal approach and landing maneuver. The landing is divided into 
13 scgmenU. The greatest density of tasks occurs during the descending task segment, TJie continuous tasks 
which have dotted Unes underneath their straight line extensions are those tasks which involve the 
monitoring of some continuous variable and the .application of a discrete control function. These include 
trim tasks and ground break tasks. During the downwind segment there are^several discrimination tasks 
which sequentially follow each other. However, these do not constitute .individual tasH segments because, 
except for Task 29, they do npt mark the ending of a continuous task. 

Table 2. Left Turn Approach and Landing Pattern Tadcs 

Function Segment: InKlal: , . 

Task Segment: Entry * ' , 

, 1 . Establish level flight at 1 ,000 feet from holding pattern before entering initial' approach. 
2. Establish an airspeed of 200 knots. ^ 
^^2^.^ Monifor^^and control the tluottle, pitcK and roll so as to maintain a constant airspeed until pitchout is 
"^>'initiaterf.^ ^/^^ ' ' ^ 

4: ' Monitor and' control the pitch so as to maintaih a constant altitude until the fmal turn is initiated. 
5. *^isually clear the area. , 

Task Segment: Initial ^ 

6., Monitor and control airc;aft functions so as to turn onto the. initial approach so that the ground track 

is aligned with the runway ccnterline . 
7.*^ Make the required radio call.* - • * 

S. Select specific ground references which reflect the rollout (at end of landing sec Figure 4) and 
^. pitchout points„at one half mile from the end cf the runway, and 3,000 feet to halfway down the 
runway, respectively. 
9. Visually clear the area of traffic. 

10. Establish and discern a ground track rendezvous with the reference pitchout poiiit. 



Function Segmtnt: Pitchout: 

■i ^ 1 



Task Segnient: Pitchout 

tl. Blend, aileron, rudder and elevator pressures establishing a 60** of Ijank left turn. • ^ 

12. "Monitor and control the turning rate so as to attain a 180** reversal in heading, parallel with the 
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Table 2 {Contma 



Function Stgment: Pltchout^ 



13. Retard the throttle to 55% simuhaneous wit HKe pitchbiit. 



14. Monitor and control the airspeed to attain a mininiuni of 120 knots by initiation of the final turn. 

15. Monitor and trim pitch and roll so as to-relieve^ undue^pressurcs. 

16. Establish and discern a 60*^ bank, | * ' ' ; * - 
17C Maintain a 60*^ banking tum . 

J8. Establish and discern the appropriate near completion oi^the turn.. 

19. Blend aileron, rudder and elevator pressutes to rollout completing 180^ of turn,^ 

20. Establish an^ discem a straight and level flight after a IBQ*^ tym. 

\ - - 

' ^ Function Segment: Downwind: 

Task Segment: Downwind 

21. Select a horizon reference point to aid in runway parallel flight,, \ 

22. Monitor and control pitch, roll, and yaw to maintain a flight path parallel to the runway using the 
reference point as a nose target. ; " ' ' 

23. Extend the speed brake. 

24. Monitor the airspeed and"discem the velocity loss to tSO kiroti. ' 

25. ,At 150 knots put the gear down. 

26. Check that gear is down and locked. 

27. Locate the final turn rollout point. * 

28. Immediately before reaching a position opposite the rollout point, place the flap handle clown. 

29. Establish and discern a ^ound track opposite the rollout point 

Function Stgmtnt: Final Turn: ,4^0 



Task Segment: Final Turn , c 

3^0. Immediately opposite the rollout point, blend aileron, rudder and elevator pressures to enter into a 
45*^ descending bank. •/ 

3 1 . Monitor and cpntrol the descent so as to attain approximately 1 00 fpm vertical velocity. 

32. ^ Monitor and' control the descent and tum sd as to attain a 700 feet loss in altitude at 180^ tum, over 

the rollout point and aligned with the runway. . / 
"33. Monitor and, control the airspeed to hold 1 10 knots. ^ ^ 

34. Trim pitch and roll so as to relieve undue pressures. 

35. Establish and discern a 45° descen|iing bank. 

36. Maintain a 45° descending tum. . 

37. Make the required raSio communication.*. ^ 

38. Establish and discern the appropriate near completion of the tum. ^ * 

39. Blend aileron, rudder and elevator pressures to rollout, completing 180° tum. 

40. Establish and discern a 700 feet loss in altitude at 180° of tum in a level and falling trajectory, 
aligned with the runway when the ground track rendezvous with the rollout point. 

^ ^ / Function Stgmtnt: Glide ftithc , > 

Task Segment: lanal'Approach ' ^ \ . , , • 

4 1 . Main tail alignnjent witinhe runway . ^ ' 

42. Maintain ^ smooth constant gHde path by controlling throttle and pitch so that the last 300 feet of 
altitude is traversed when the aircraft reaches the .touchdown point. / 

43. Place the end of the runway (aim point) in the center ofjhe wina$aeen^and maintain it at this 
position until foundout. * . • 

4l. Contrpl the throttle to allow the airspeed to decrease to JOO knots. 

45. Establish and discem the desired airspeed coincident with a position just in front of the aim point. 



Tabk 2 (Continued) 



Function Stsmtnt: GlldtPatti: 



Task Segment: Roundout . • ^ 

46. Immediately before the aim point is reached apply baclc pitch pressure to hold the aircraft just off the J 

; . runway. . ^ * • • ^ ^ 

47. ,Gradually reduce the power to idle and decrease the airspeed to 75 knots.. ^ - ^ 

48. Establish and discern aircraft holding position just off the ruriway. - ' ' ' 

Task Segment : Touchifowti ^* . . ' , ^ 

49. Control the pitch to smoothly touchdown the rear wheels. ^ 

50. Establish and discern rear wheel touchdown. 

51 . Controf the pitch to*smoothly lower the nose wheel to the runway, 

52. Establish and discern nose wheel touphdown. ' , * 
•53. Retract the speed brake. 

54. Check rudder pedals to neutral. ** 
' 55. Engage nose wheel steering. 

56. Conlrol and monitor nose wheel steering, ' ^ • 

57. Apply brake pressure as required to slow to taxiing speed. 



The major control functions Fulfilled by the pilot for the normal approach and landing pattern are 
^oll, pitch, airspeed, altitude, and ground track. Of these, ai^pee^ a function of RPM and other flight 
control functions. Perceptive process functions monitor and/ analyze the conesponding flight variables 
which reflect ^hese control functions and determine their impact upon airspeed. Airspeed, therefore, 
becomes th? focus of functions, and is dominant for certain segments. A similar rationale also appli^ for 
the altitude control function. *^ , ^ ^ 

.Minor control functions required during a normal approach and landing maneuver are trim, rate of 
descent, nose wheel steering, wheel brakes, RPM, yaw and heading. These continuous functions are affected 
by the application of the discrete control functions of speed brakes, flaps and gears. For the purpose of 
analysis, the trim and,the wheel brakes control function is included in the continuous control functions due 
to the nature of the continuous monitoring tasks which must be performed in order to utilize these discrete 
control functions. * * 

The dominant control functions required for^the vanous functional segments are as follows: 

1 . The initial functional segnlent has the dominant control functidns. airspeed, aUittide, and ground 
track.* ^- * * 

2. The pitchout functional segment has the dominant control functions: roll, phch, airspeed,^ J 
altitude, and ground track.* ^ » 

3. The downwind functional segment has the dominant control functions', pitch, airspeed, altitude, 
and ground track.* * , ^ ' 

4. The final tum functional segment has dorrtnant control functions, roll, pitch, airspeed, altitude,, 
and ground track.* . . ' ' * 

5. The glide path has dontinarrt control functions: oitch, altitude, airspeed, and ground track.* 

6. The rollout functional segment has grounji track* 6s its domin^t control function. " ^ 



^Ground track is not measured directly in the aircraft. It may be possible to use heading m^ormation in conjunctiOR 
with an estimate' of Avmd forced and an aircraft model to-estimate ground tr^ick. ' 

*♦ * 

' . . • 23 >- ; • - " 



Figui€ S relates various variables and segments for the Normal Landing maneuver in the appropriate time 
domain. During the initial function segment, the dominant control functions do not reflect variables which 
maintain significant amplitude^ in their variations. Therefore, the initial function segment can be considered 
as an event sequence^ i.e., measured with respect to an end condition tolerance. On the other hand, the 
pitchQut functional seaprtat does maintain significant amplitudes in the variation of their respective flight 

* Variables. Therefore, r^/U aiid airspeed are candidate ineasure variables in fhat segment. Thus, the pitchout 
functional segment is*^ treated as a locus segment. On the other hand, the downwind functional segment 
maintains a signifidant variation of only one dominant cdhtrol function (airspeed), therefore, it is useful to 
consider the downwind as 'an event sequence which is ordered with respect to airspeed. The final turn 
maintains significant amplitudes in the variatipns of control functions roll, pitch, and altitude. In this 
instance, the rate of change 6{ altitude is a function of the pitch. Therefore, it wduifd be efficient to exclude 
either the altitude or the pitch from the measurement variables siiiice one of these reflects the otjiej. Since, 
altitude i^ the net end objectivp of pitching,* the pitch is dropped in favor of the altitude measurement 
variable. The glide pathv qm irtclude the tasksx)f roundout, touchdown and nosedown if these tasks can be 
measured. Pitch is considered as a measurement' variable along with altitude and airspeed. As before, 
altitude rate of change is a function of pitch and other variables, but, glide path errors can be analyzed as an 
independent variable. Finally, the rollout is analyzed as ah event sequence in time . 

Banc! Roll 

Table 3 lists the tasks involved for the performance of the Barrel RoU maneuver. W Barrel Roll tasks 
are similar to those of the Lazy 8^ maneuver. They both require a perceptual task to form a specified 
geometric figure vnih the nose pirpjecti9n upon the horizon. Figure 6 illustrates the fiEarrel Roll wkh its task' 
segments" time charted. In this case, there are two initiating task segments leading up. to the Barrel Roll 
maneuver segments. These arc termed "initial" and "entry." Both segments contair\^continuous feedback 
tasks terminating with critical discriminations. Note the overiapping and sharing of thtf same time space of 
Tasks9, 10,11,12, 13, and44. ' 

The Barrel Roll can be divided into two partitions of consistent.cojntinuous control function domains. 
These segments are termed "initial" ai^ the "barrel*' The-initijd^iicludes two t^sk segments (initial and 
entry), as discussed previously. The tek&oh both segments arc 'considered within the same functional, 
segment due to, the shared objective of entry attitude. This consideratioa allows the treatment of the 
initial function segment as an event sequence ia time. The initial function segment has dominant control 
functions; airspeed, altitude and heading^ Its minor control fuactions are yaw, pitch and roll, During the 
initiar segment, the pitch seems to take on the major fimction role. However, the objective of the pitch 
down phase of the initial segment is the attainment of a specific airspeed. This attainment of airspeedis so 

^ specific that it can therefore h[e clearly separated from the primary portions of the barrel roll maneuver. 
The same argument applies for the roll during the entry task segment . - , ' 

' ' ' ' ^ . * * • ' ' , 

^ , r^Wei. BarrtlRoILTasks . * * 

^ \ . > Functlbn Stament: tnitUi: *- 

Tadc Segment: Initial \ , * * ■ 

* a. ■ Establish level fligiit' at 8,00dfeet» . • ' * 

2. Select a rjsferenqs point oh or rieaf. the horizon. ^ ^ ' 

3. Visuallydcar the area of all aircraft. ' •* ; • - ^ 
. 4. Set power at 90% RPM. ^ ' . k; y 

5. Enter a shallow dive with the nose of the aircraft below the refetencc point in order to attain an air 
' .speed Of 230 knots*^ ^ ^ ^ ; \ ' 

6. Establish and discern an airspccdof 230 knptS. • t ■ * • ' ^ ; 

Task S^ment: E^tiy . " , . ' *^ * • 

/ 7. Begin a coordinated turn in the opposite dircctibn of the dejjired roll in order, to rais^tiie nose tq the 
horiiSbn at\ wingi-leyel attitude in,a direction 20"* - 30** to the side of the refereiice pobit. 
8. Establish and discern a level flight in a direction 20"* -^^30^ to the side of the fefercnce point. 



Table 3 {Continuetf) 



FiincUon Seiment: B^rrtf: 



Task Segment: First Quadrant 

9, Monitor arid control the aircraft flight parameters such that the nose of the aircraft describes a circle* 

around the reference point on the horizon. • . * . 

10 Immediately and smoothly raise the nose while gradually blending in aileron and rudder pressure 

/entering a dirobirig roll, so that n^^um pitch rendezvous with 90"* roll. 
1 1 . Control ailerons so that a constant ralS' of roD is maintained throughout the maneuver. 
M2 Control pilch, r^ and yaw such that the angle between the reference point and the nose'projection is 
kept constant throughout the 'maneuver. ♦ 

13. Coordinate the controls so that minin}um airspeed coincides with inverted flight, not tp decreaser 
below inverted stall. ^ . 

14. Coordinate the controls so that maximum altitude coincides with inverted flight. 

^ 15 Establish and discem a maxiirium pitch equal to the original offset an^e when a 90"" roD is attamed, 
thus describing one quadrant of the projected^circle. 

Tadc Segment : Second Quadrant 

16 Relax some of the bade pressure while continuing the same rate of roD by blending in more aileron 
pressure so as to attain a 180^ noU al zero pitch- w}th an offset angle equal to .the initial but to the 
opposf^side of the reference point, with the minimum airspeed and maximum altitude. 

17. Establis£W discern a 180"* roll at zero pitch with an offset angle equal to the initial but to the 
opposite-^Se of the reference point, with the mininHim airspM aad-maximun»-aiytude~thus 
describing the top half of the projected circle. , . ^^ » 



Task Scgn>ent( Third Qxiadrant 

18 Be^ applying increasing back pressure while continuing the same ratetpf roll by deaeasing aileroa 
^ pressure so that the nose of the aircraft describes the third quadrant of the ^rojeqted circle belo^ the 

horizon cubninating with a maximum negative pitch rendezvous vntMH^f of roD. 

19 Coordinate the controls so that mnimu^n ;pltitude equal to that of the initial low altitude, comades 
with level flight. , 

20 Estabfish and discem that the maximum negative pitch coincides with 270*" of roll, thus describmg 
three-quarters of the completed circle. 

Task S^inent: Fourth Quadrant ' ^ 

,21 Increase back pressure and decrease aileron pressure so as to attain a 360* roll with a zero pitch at an 
offset angle the same as the initial. * , 

'22 ^ Establish and discem a level flight at the original offset angle to the reference point thus descnbmg 
the completion of the 360"* nose projected circle about the reference point. 

During the barrel function segment, pitch and roll arc the dominant control functions. "ITie minor 
control functions are yaw, altitude, airspeed and heading. . 

The dominant control functions are shown as a fiinction of time in Figure 7. Htch and roUare seen to 
maintain significant amplitudes in their variation and arc continuoudy variable throughout* the enure 
maneuver. H^rcforc. the. barrel fitaction segpient fcan be considered jas a locus segment described by a 
funetiooal relationship between roll and pitdi. ' " 

Splits' * ' ' 

Table 4 lists the tasks required for the performance of the Split S Maneuver. The Split S is a recovery 
maneuver from a position of near stall vrfth 90 percent of the engine power. There are 17 tadts involved 
the performance of this maneuver. These, tasks arc divided into 4 task segments, initial, entry, pull through* 
and exit These task segments arc in turn subsets of the two function segmejits, the initial and the half4oop! 
The final objective of the initial function segment is to pitch up and roll'tp inverted so that a pull through 

■ . ; . 28 ' ' • • 
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TCi^very can be initiated.^ The time sharing bf these tasks is represented in Figure 8.' Task difftculties are 
increased at tlie entry pomt as shown by the time sharing of Tasks 10 and i L This difficulty of sharing 
time domain between varioos tadcs continues thxoughoat the entire half4oop function segment. Note that 
the rott to iilverte^ B a continuous feedblack t^ occupying very little time. Its objective is to "attain an 
attitude for puD through 'execution.. Therefore, it' is considered as an event in the event sequence of the 
initial function segment. The saine hoids true for th^ pitch up at the start of the dimb. 
, ^ * ' * 

' ^ ' Table 4, SpHtS Tasks 



. ^ FurtcUon S^gmtnt; \t\\\XtU 

•Task Segment: Initial ' ' . ' 

i 

\ Establish level Qi^t'at a sufficient altitude so that the.^titude loss incurred iri the maneuver does not 
cause a penetration hdow 5,000 feefahitude. ' " ^ 

2. Set power at 90% RWrf.. * 

3. ^ Visually clear the ^a of all aircraft. 

4. R^chl^p25^ * % 

5. Monitorthe airspeed to ascertain a decrease to 120 knots. 

6. Lower thp spjped brake if tfc airspeed is excessive. * 

7. Establish and dispem a 120 knot airspeed. ' 

8. Retract the speed brake If used.' 

9. Roil the aircraft 180**. 



Function S«gmtnt: Half Coop 

TasJTS^grartrEfltiy ~- ^-S. . ^- 

10. Execute a half-loop to attain a re versal in heading arriving at an erect and level attitude. ' 

11. Slowly apply back pitch pressure brin^g the nose of the aitcraft throu^ the horizon in order to 
attain the maximum b^ck pitch pressure possible without stalling. . * 

12. -Establish arid discern a maximum pitch hack pressure without stalling. 

Task Segment; Pun Through . 

13. Monitor 'and controt the acceleration to insure that the 'G* limit is not exceeded through ^e pull 
through. . - * ' 

14. Hold a maximum back pitch pressure. 

15. Establish and discern the appropriate near completion of tj^e puU through. 
Task Segment: Exit 

V6 Release bacTc pressure as the pull throu^ nears completion to Mtain level flight in a reversed heading. 
17. Establish and discern a reversal in heading at a straight and level attitude. 



The continuous control functions altitude, airspeed, pitch, and roll are dominant in the initial 
function segment. « a', 

Miiior continuous control functions are yaw and Heading. Rtch and acceleration are dominant control 
functions for the half -loop function segment., The minor functions are altitude, airsp^, and headmg. 
Figure 9 illustrates the dominant measure variables.as a, function 6f time. As shown, they further mamtam 
significant amplitudes in their variations and their first time derivatives consistently sustain /ipn-zeio values. 

Oovcricaf . ' 

The Cloverieaf maneuver is broken into component tasks as sh6wn in Tablets. It shows the parallel 
relationship to thit Split S maneuver. As with the other maneuvers, the first function segment establishes 
the entry attitude for the aircraft. The climb function segment follows the initial and is the entry into the^. 
loop portion of the maneuver. Pitching up until 45^ pitch^gle is attained, the pilot enters Jnto the ascent 



turn function segment which is the iirst of the 4 similax loop function segments to be employed. The ascent 
tunU>asicaIly accompJishes„a_xoll,tQinYerted along, with.a ^O'* phangg in Jle^ading, TheJialf-IoppJimctlon 
sclent is a puj throu^ similar to that of the /Split S maneuver except instead of exiting at a level flight 
attitude, flic pU(jfaout is continued by raising the nose through to another ascent turn functional segnient. 
The repetition of these two functional se^nents continues until the fourth and final doverlpaf loop is in Its 
half4oop jiiast. At this point in the maneuver, the puE through is scheduled for termination just before the 
strait ^d level attitude is attmned. The rate of* pitch change is diminished to zero instead of following 
through with a continued pitch rise. This acQomplishes an effective exit from the maneuver at a strai^t and 
level attitude. 

^ 1 JTablaS. Cloverlcif Tasks - 



Function Svfmtnt: InltUI: • 

Task Segment: Initial , ' • - - . 

1. Establish levelflight a/ 8,000 feet.' . - / ^ 

2. Set throttle t6 95% RPM. . • ' » ' 

3. Visually clear the area"of aD aircraft. , 

4. Select 4 reference points on the horizon, one to. the fropt, one to the rear, and one off of each wing 

tip- . r : . . . . ^ 

5. Enter a shallow dive to gain sufficient airspeed so. that as the aircraft is returned to level flight, an 
entry airspeed of 220 knots is attained. ' - 

Est^Ush and discern a level fHght at 220 knots. ^ ^ 

Ftmctlory S t m > nt; Cllmbt ' 

T»k Segment: Pitch Up ^ « 

7. Perform four equivalent twisted loops, turning 90° from.entty heading^at 45° of ascent pitch in such 
a maimer as to form two concentric crosses on the groimd with the nose projection. Thf cross tips 
should terminate at the 4 reference pomts. * % 

8. Immediately following- the 220 knot airspeed attainment, continue smoothly exerting Back pitch 
pres^^ to maintain a constant rate of pitch m an ascending loop so as to attain a 45° pitch attitude. ^ 

9. Cooromate the controls so that minimum a^speed coincides ^th maximum altitude at inverted fli^t 
at the top of the twist loop. 

10. Monitor and control the loop turning rate so. that level attitude comcides with 90° to the right diange 
in heading at the end of the loop. 

11. Established discern a 45° pitch attitude with zero roll. 



Functto;i5«|nMnt: Asetnt Tumi 

Tide Segment: Turn 

12. Blend in roll to execute a 90 left ascending turn attaining an Inverted attitude directly heading at the 
left wing reference point'at the top of the first loop. 

13. Establish and discern a 90° left turn, inverted attitude,.maximum altitude and minimum airspeed at 
the top of the fir^^ twistedloop. 



Fuhcilon ScfrTMntx Half Loop 

jMk Segment; Pull Through. . • 

14. . Execute a loopito attain a reversal in heading, arriving at a 45° pitch with a zero»roll. 

1 5 . Control the aircraft In such a manner as to point a straight line from the ori^al left wing tip horizon 
, reference point to the rigjit wing refereiice poirit with the aircraft nose projection. This line will be 

perpendicular to the ori^al Hhe of travel* ' - 

16. Momtor and control the pitch to hisure that the *G* limit is not exceeded throu^ the puD through 
- and^a constant rate of {dtch change is mai^itained. _ ' 

17. Establish ^d discern the right reference point m the center of the windscreen at level flight 
completing one perpendicular Kne projection and completing one loop. 



Table 5 (Continued) 



' , - -V Function StsmtnttHilf Loop , ' ' " ' 

Taak S^ent: Pitch Up 2 ' . ^ 

18. Coordinate the control so that mininaum airspeed coincide with maximum altitude at inverted flight 
at the top of the twisted loop. , ^ • * 

19. Monitor and control the loop turding rate so that level attitude coincides with 90° to the right chajige 
in heading at the end of the loop; 

20. Establish and discem a 45** pitch attitude with zero roH • 

Function Stflmcnt: Asc«nt Turn 2: . ^ ^ 

Task Segment: Turn 2 • ^ 

21. Blend in roll to execute a 90** left ascending turn attaining an inverted attitude directly heading at the 
forward reference point at the top of the loop. , 

22. Establish and discem z 90** left turn, inverted attitude, maximum altitude and minimum airspeed at 
the top of the second twisted loop. 

Function St^mtnt: Half Loop 2: < ^ . ' 

Task Segment: PuU Through 2 

23. Execute a loop to attain a reversal in heading, arri^ng at a 45*^ pitch with a zero roll. . 

24. Control the aircraft in such a manne; a? to point a straight line from the original forward horizon 
rcieiuiia; poml io Uie rear reference point with The aircraft nose projection. This line will be parallel 
to the original line of travel. ' ^ . ^ 

25. Monitor and control the pitch to insure that the *G* limit is not exceeded through,the pull through 
and a constant r^ of pitch change is maintained. ^ . 

26. Establish and discem the rear reference point in the center of the .wjndscrcen at level flight 
completing the parallel line projection, the first groxmd cross, and the secoA^J.loo'p. 

.-iftsk Segment: Ktch Up 3 . . 

27. Cpordinate the controb so that minimum airspeed coincides with maximum altitude at inverted flight 
at the top of the twisted loop. 

28. Monitor and control the loop turning rate so that level attitude coincides^th 90*^ to the right change 
in heading at the end of the loop. ^ ' r 

29. Establish and discem a 45'' pitch attitude with zero roll. ' * 

* FuncUon Stflnicnt: Asc«nt Turn 3: 

Task S<s?nent: Turn 3 

30. Blend in roll to execute a 90** left ascending turn attaining an inverted attitude directly heading at the 
right reference point at the top of the loop* 

31 . Establish and discem a 90** left turn, inverted attitude, maximum altitude, and minimum airspeed at 
\^ the top ofthe third twisted loop. ^ ' - 

^ ^ - Functidn Snn>tnt; HUf Loop it 

Task Segment: PuB Throu^ 3 ^ • - 

32. Execute a loop tp attain a reversal in heading, arriving at a 45** pitch witl^ a zero roll. 

33. Control the aircraft in sudh a manner as to point a straight line from the ori^al right horizon 
reference point to the left reference point with the aircraft nt)se projection. Tliis line will be 
perpendicular to the original line oftravel. ^ . 

34. Monitor and control the pitch td^insiire that the *G* HmiVis not exceeded through tiVj^ through and 
a constant rate^ of pitch diang? is maintained. ^l^Kc^ 

35. EstabKsh and discern the left teferencc point in the center of the ^dscrecn at level fli^ifwmpleting 
the perpendicular line prqection, the first axis ofthe secondcKSs.and the third Joop. 
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Table 5 (Continued) 



Function S«gmtnt: Half Loop 3: 

Tadc S^ent: Ktch Up 4 

36. Coordinate the controls so that minimum airspeed coincides with maximuin altitude at inverted flight 
at the Top of the twisted loop. 

37. -^ Monitor and control the loop turning rate so that a level attitude coincides with 90*" to the right 

change in heading at the end of the loop , ] \ 

38. Establish and discern a 45^** pitch attitude with zero roll. 

Function Stgmcnt: AK«nt Turn 4: 

TWc Segment: Turn 4 ' . . " ' 

39/ Blend in roll to execute a 90** left ascending turn attaining an inverted attitude directly heading at the 

rear reference point at the top of the loop. 
40. Establish and discern a 90" left turn, inverted attitude, maximum altitude, and minimum airspeed at 
" the top of the fourth twisted loop. 



Function Sesmtnt: Half Loop 4: 

Task Segment: Pull Through 4 / 

41. Execute a loop to attain a reversal In heading, aniving at a zero degree pitch with a zero roll, headed 
in the original direction. 

42. Control the aircraft in such a manner as to p oint a straight line from the original rear horizon 
reference point to the forward reference point with^the aircraft nose projection. This line will be 
parallel to the original line of travel. 

43. Monitor and control the pitch to insure that the tJ' limit is not exceeded through the pull through 
and a constant rate of pitch change is ifiaintained. 

44. Establish and discern the appropriate near completion of the pull through. . 

Tadc Segment; Exit 

45. Release back pressure as the pull through nears completion to attain a level flight in .the original 
heading. 

46. Establish and discern the forward reference point in the center of the windscreen at level flight 
heading in the original direction completing the last parallel line projection, the second ground cross 
and the fourth loop. .* ^. r ' 



The composite tasks involved in the accomplishment of each of the function segments are 
schematically drawn in Figure 10, the Clovcrleaf Time Chart. As sTiown, the repeated structure in flight 
tasks begins during the first ascending turn. This in turn is duplicated during the subsequent ascending turris^ 
two, three, and four. ^ 

The dominant continuous control function utilized during the initial function segment are the 
functions of airspeed^ jieading^^ pitch and altitude. Roll and yaw are of relativel]^ minor importance during 
this functional segment^ During the climb, pitch is the single major control'function while heading, altitude 
and airspeed take on relatively minor importance. Htch and roll are the dQminant continuous control 
functions for the ascent turns. Heading, altitude, airspeed and yaw are supportive control functions wliich 
aid specifically in the discriminatory tasks. Rtch and acceleration con^i^l are dominant during the 
half4oop. The minor functions are heading, altitude,*and airspeed. The relationship in time between these 
dominant control functions is shown in Figure 11, which relates the measurement variables^^ function 
segments and various flight parameters for the Cloverleaf maneuver. The initial and climb function segment^ 
can be treated as, -a single event sequence in time. Thus, proficiericy for that segment can be measured by a 
test which d6terntiiie|^variable values are within tolerance limits at the end of the segment. 



V. SEGMENTAHON LOGIC ^ . ' 

Introduction . < * • 

. • • ' ^ • r> 

Tl^e functidn and tadc segmentation, as d'scusscd in Section IV, is implemented in the computer 
processor by a sej of segmentation Jogic functions. This logc automatically divides each maneuver into the 
desired segments by identification of the proper start and stop conditions for each segment. Detailed logic 
equations are' presented in (Connelly et d., 1974). In this section, we present arbncf djjscriptjon of cues 
used fot detecting task segment boundaries. * ' , ^ 

Atttom^c Segmentation Logic^ ^ • . ^ • 

The segment num^igp and nanTjfes for the Lazy 8 are shown in Figure 12..The processor automatically 
determines a new segment upon idei^dfication of the proper segment start.condition. Thfe end condition for 
a segment is the start condition of the next scgrient. The general logic for the Normal Landing, Barrel Roll, 
Split S, and Qoverleaf is shown in Figures 13, 14,. 15, and 16, respectively. • 

VL GENERATION OF Mi^EtJVER REFERENCE CRTTERION , ^ ^ 

FUNCTIONS BY REGRESSION ANALYSIS ' 



ReferenceFunctions . 

Iif order to develop candidate profidetlcy measures it is fljst necessary to establish a refereiicc 
(criterion) maneuver trajectory or.trajectories for eaph itiancuver type. This .reference is to be used for 
comparison with student flig^its in order to implement error measurements. Gener^y the required leference* 
functions are developed from maneuver specifications; however, the maneuver specifications do not 
identify a unique reference trajectory. The question then is; "b there more than one way to fly th^ 
maneuver and achieve excellent performance?" If so, we then need to, know if there is a manifold of 
acceptable trajectories or are there clusters of trajectories which can be distinguished. One reason a 
manifold of trajectories may appear when examining high performance data (lie., fli^t performance rated 
excellent) is that trajectpry parameters are required. For example, the form of the reference trajectory may 
be a function of initid aircraft states (attitudes, airspeed, and altitude). Likewise, there is need to 
investigate the relationship of proper flight in one segment to that achieved in another segment. For 
example, the second half of the Lazy 8 is to be an image of the firSt half of the Lazy 8; therefore, the 
reference maneuver for the second half of the maneuver is based on flight patterns produced in the first 
half. , • 

The*approach used here is to examine flight data from those flights rated excellent and determine if 
these flights satisfy the deifications and are dosely grouped together. The specific question Xo be 
answered is: "Does the average over these test flights represent a suitable reference maneuver?" The 
variance of thesd test flights is used as one method of judging whether or |iot the average trajectory is a 
suitable reference trajectory. For example, if all the high performance te'st flights are grouped closely 
together, ^,t^ Svith a small variance compared to the variance obtained from. the other performance 
extreme; i.e., neophyte student pilot flight^^-the "average of the high performance flights can provide a 
suitable reference maneuver. An illustrative example of this situation is shown in Figure 17. If it is shown 
that the high performance flights show several dusters, a reference function can be constructed for each 
cluster.'^ 

A further possibility i§ that a manifold of high performance fli^t trajectories may appear, all 
members of which satisfy maneuver spedfications. In such a case,tdl these maneuvers are satisfactory, but 
one is to be prefen^d since it also satisfies some addition criterion (such as mi^mum "flight smoothness"). 
In this case, there is a region of accept^le performance and pne trajectory in that region is recognized as, 

superior. " ^ , • ^ " 

* ' ' 

An alternative way of specifying a riianeuver reference is .to not seek an isolated reference trajectory, 
but instead seek a reference rdte of change of each critical flight variable wfthin the flight envelope. For 
example, we would seek to represent the desired rate of change of pitch angle as a function of pitch, roll, 
yaw, airspeed and perhaps a maneuver state indicator in hm of expressing pitch directly as a function of 
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-aixsRc^d, roll, etc. The reason a variable Kite of change as a function of the aircraft fHght variables is used is 
IhatAc integral fonn (and associated M^M.conditipns) is not regi^^ of constraming the 

foimulatioa to representing a specific reference in a given function space, tiie rate expres^on is used so that 
the reference maintains a general applicability no matt^er where the flight path initiates. An exainple of this 
method of representing a reference maneuver is shown in Figure 18. It is seen that v/ith this approach, an 
isolated trajectory reference is not acquired. It is also observed that hi^ performance flight data over the 
maneuvfir envelop is required. . * . . ^ . » , 

Regitek)n Analysis Tedmique , . / 

A least squares regression analysis was performed oh excellent rated performance demonstration data 
to generate ,the desired reference functions for each maneuvi5r segment. The specific functions that were 
generated ate defined in Section Vn of this report. 

The regression analysis is performed in an iterative way so that new d|(ata can easily be added as it 
becomes avaihljle. The method is discussed fuUy in Connefly et d., (1974)^ 

VIL CANDIDATE MEASTOES 

General " ' . . 

As discussed m Sections IH and IV, maneuvers are partitioned into function segments and task 
segments. A function segment is, that portipn of a maneuver in which the set of dominant measurement 
variables is consistent. A task segment is^ that portion of a function i^gment'in which the relationship 
among dominant measurement variables is consistent. Pilot tasks required in each function segment can be 
further categorized as locus (continuous) segments, and sequential segments (e>ent tasks) where the locus 
an4 sequence segments are not necessarily mutually exclusive. For example, several discrete tasks may be 
* required 4n a lotus segment. Thus, a portion of a maneuver could be simultaneously treated as a locus 
segment, as well as a sequence segment.' Evaluation bf pilot performance during that portion of the 
maneuver would/necessarily be a combination of the separate locus and ^quence segment evaluations 
Fmally, it is noted that continuous operations can be treated as discrete events provided a threshold is 
establishexlto~define~aregion'of acceptable-picrformanw^ — — - 

Locus (continuous) and sequence (event) segment task categories identify two of pilot tasks. 
Within the set of system tasks of which pilot tasks are a subset, an additional task* category of system 
output tasks can be identified. This category inchides thosp tasks which can be directiy measured with the 
aircraft's fK^t variables. These aret * ' 

a-. Establishment tasks which establish a specified condition, for example^ altitude*, heading, or rate 
x)f.tum. ^ ^ ^ . ' ' ' ^ 

b. Maintenan'ce tasks which maintain a constant condition such as constant altitude. . 

c. Recovery tasks which recover from a fli^t error condition, for example, recovery from a glide 
^ath error. « • ^ » ' . 

d. Coordmated tasks wIBeh conduct a^coordinated maneuver where dhe or more control variables 
are coordinated.with a reference variable, for example, a cHmbing turn. 

Continuous task, types (b) an4 (d) are Usually associated with a reference or criterion functibn >^ch 
provito a reference flight trajectory. These reference trajectories are part of the maneuver spccificition. As 
indicated, m.ancuver spdcifications do not always define a unique aircraft tr^*ectory. In some cases there 
.may be more then one way to fly tiie maneuver in order to satisfy the spedfications. Therefore, ^ther 
many fH^*t trajectories are acceptable mdtt the maneuver specifications; i.e., there are re^ons of 
acceptable flight trajectories, or, there are additional criterion functions (for Example, **flight smoothness") 
that can be used to select a preferred trajectory from those vAdch satisfy the maneuver criteria. Thus, while 
a manifbld of trajectories is acceptable, one of the trajectories may bi5 preferred if it best satisfies additiorl^ 
criteria, * 

* * 

Reference aircraft trajectories and maneuver specifications are generally not available for task types 
(a) and (c). Perhaps this is because there are njany possible iiiitial conditions for establishing a specified 



condition, likewise, the number of possible enor recovery situations is large. Anotfier reason for the l^ck 
of specifications for recovery tasks may be thajt methods of defining the recovery specifications may not 
have been apparent. ' ' . ' , 

* « ' I- 

Measure Definitions and Notation * 

a. Continuous Flight Tasks. Some continuous flight tasks, sudi as those requiriog maintenance of a 
constant condition, or achieving a coordinated maneuver, can be associated with a specific reference 
trajectory. Because of this, candidate proficiency measures' can be constructed with respect to this reference 
trajectory. An ejror function can be, constructed as a differwice between the priterion reference function 
and the actual flight trajectory as shown in Figure 19. The candidate .measure could be defined as a 
function of, periiaps, the absohite value of that error measure. 

It is also possible to construct a tolerance about the reference function which. reflects the region of 
acceptable flight, as shown in Figure 20. This tolerance need not be constant and, indeed, can vary along 
the flight path. In this case, small variations about the reference flight path are ignored until the actual 
flight path exceeds the tolerance vvliereupon an error event is generated. Thus, proficiency measurement of 
flight tasks for which continuous references are available, can be treated by either continuous error 
functions or by discrete error events. 

An altoftiate wgy of representing the reference of a desiredtnaneuver is vi^ 'a difference equation. This , 
equation specifies the desired rate of change of each flight variable of interest as a function of state 
^variables: ' , * 



AXj = A)COCp = AX.(X, X„) . ■ 

where X. is a state variable. 

A reference (criterion) maneuver can be considered as a vector with coitlbonents corresponding to the 
desired rate of change in the state variables. This is illustrated in Figure 21 ^ the simplest case of a two 
space representation. The vector difference between the reference and actual vectors is expressed as 



ERIC 



where ^ • < . , ' • 

D = difference vector lejigth e . \ 

Ki weighting^unction for the state variable Xi rate of change ^ 
' K2 = weighting functfon for the st^kte variable X2 rate of change * , , 

The Kj functions are included in order to provide for a sufficient degree of generality in the application^. In 



general, the vector difference will be expressed as: 



where 

.D =D(X) ; ' . ' ' , 

In such a case, the distance from the actual ^vector projected to the reference vector may be the salient 
variable. Let us represent this error vector and its length as 

E. = D;Sin8 . ' ' . ' 

where " * • ' 

, E is the error vector length and 5 is the an^e the difference vettor makes with the reference vector. 
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b. Dmrete Fiiiht Tasks. Discrete tasks are associated witH either a continuous variable where the 
task, is to tie accomplished .within ccrt^. values of that continuous* variable, or they are associated with 
other discrete events where the event is/to occur in some specified sequence. If the event is referenced to a 
continuous variable, two types 'Sf nWuir^ are possible. One type of measure is a Boolean measure which 
indicates whether or not the event took place within the prescribed limits of the continuous variable, as 
shown in Figure 22, If the event docs not occur within that prescribed interval, an enor measure indicating 
the extent ,of out of tqkrance may be useful. This type of enor measure is illustrated in Figure 23. Figure 
24 illustrates the measurement of sequential enor where we require that the event under test occur after 
event A and before B. This leads to a Boolean measurement that the correct sequence has occurred. 

Proficiency Measurement Concepts < 

The types of enor measures identified above can be used in several ways. The way that an enor is 
measured can be selected based on whether that enor is the primary or secondary concern in measuring 
proficiency. For example, the primary errors Jnight be measured with the continuous difference method 
(Figure 19) and secondary or lesser important enors measured with the threshold technique (Figure 20). 
The combined measure would consist essentially of the continuoas factor unless the threshold wafc exceeded. 
Therefore, the combined measure^ would emphasize the primary error factors .iid de-emphasize the 
secondary enot factors unless the secondary factors became sufficiently large. T^ie primary enor measures 
might be those associated with tasks .judged to be difficult or otherwise judged to be important to that 
maneuver. Likewise the secondary enors might be associated with tasks judged to be easy, for example, 
maintaining a constant value of a variable such as altitude ^r heading. 

Enor jneasurcs can also be made functions of variables other than enor terms. For example, the 
weighting of an enor can be based on whether or not the enor is diverging or converging. Also, the 
-i^gl}ting of an enor term can be a function of a discret|b variable so as to change the importance of enor 
depending on concunent activity. * 

a. Methods of Combining Measures, k task segment may contain various types of tasks. As a result, 
is necessary to combine the candidate proficiency measures from each task in order to forln a candidate 
nent measure. Likewise it is desirable to combine segment measures from each segment in the maneuver 
to^rm an overall maneuver measure. Th6re are several ways of combining the candidate measures and each 
can \e tested to determine which method provides best discrimination of extreme performance. Four 
methcm of combining segment measures are as follows: 

1 . Linear sums with the weighting terms determined by: 

a. Relative variance of the individual measures, and 

b. Capability of the individual me^asures to separate extrmf performance^ " . 

2. Selection offhe largest enor term. a . . * , 
^ 3. Discrete factor parameterization of continuous enor measures. ^ ^ v " ^ 

4. Maintain the individual enor components in vectqr form. ^ r 

The linear-sum-com^ined measure allows individual weighting'of each enor component, with each enor 
term thereby contributing to some extent to the combined measure. The measure obtained in selection of 
the maximum or largest enor term is very sensitive to one error factor and, therrfore, would be very 
responsive to the pilots' attempts to improve that flight factor. Discrete factor parameterization of 
^ continuous error measures allows a different weighting of continuous measures based on other (discrete) 
pilot action;. For ex^ple, it. is possible to wei^t the importance of, try, an altitude hold enor as a 
function of other tasks he mi^t be performing su&h a communication tasks. This recognizes that 
proficiency may be a function of how well a continuous task is performed when the pilot is attempting to 
accomplish a secondary task. Finally, the rationale for a vector performanoe measure, where each vector 
component is a summ^ measure for a type of ta^, is that the combination of performance measures from 
similar types of t;^ks provides separate proficiency measurements for each different type of task, X^us, it is 
prefened t6 cdmbine (e.g., by addition) measures of similar tasks from one segment to another. The 
resultant of this combination is a maneuver vector with components which provide a measure of 
proficiency for n different pilot skill areas. ( ' ^ ^ 



\ 



so 
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/ The n components of the vector are the proficiency measures for the n different types of f&ght 
factors. A subsequent combinational operation can yield a sin^e maneuver proficiency measure if desired 

Definition of Error Measures 

Previous sections have identified wsveral primary enor measurement ^techniques as well as several, 
possible operations on those measures. Since the initial result of the maneuver analysis is the development 
of candidate proficiency measure^ which are to be subsequeritly tested with flight data, the number of 
combinations of measures and operations on measures can be large. This could lead to a clumsy 
documentation system unless some convenient compact notation is empbyed. Thus we first must develop 
acceptable notation before operations are possible on error measures. This notation should identify 
variables, parameters, and tolerance values of interest in addition to providing the information required to 
implement the primary enor measures on flight data, 

a. Continuous Difference Error Measures^ A reference trajectory is formed as : 

5r=fj(Y,,Y,,...,Y^) ^ ' . . ' : (1) 

where » 

Xj IS a flight variable treated as a dependent variable*for the purpose of establishing maneuver criteria, 
i distinguishes this reference trajectory from others; Y|, Yj, ...,Yn are flight variables treated as 
mdependent variables for the purpose of establishing maneuver criteria, f, is the function relating the flight 
variables and defines the maneuver criterion function. Note that is the predicted vdl^e of variable X 
(deterriuned from high performance flight data) m a given segment in the maneuver, and that the dependent 
variable as well as all of the mdependent vanables are functions of tunc. An example of equation (1) would 
be ' 

?=f(<^^,AAS) . ' (2) 

The conesponding primary difference error measure asillustrated in Figure 17 is 

Ej^r |x.f^(Yi,Y,,...,Y^)|^£^^(X.Y^ Y,,...,Y^ (3)' 

where 

s^bscnpt M indicates the error measure is a meaa of difference from a reference trajectory function. 
Tlierefore, in summary the notation 

E„.(X;Y,,Y,,.;.,Yj . ' (4) 

indicates a mean trajectory critejion functionis used, X being the dependent variable, and i identifying a 
specific function. The error chosen for Em is tik/nwafiabsolute deviation given explicitly as 

E^ = l/n^2 IXj-Tll ^ (5) 

' . i=l 

This enor measure can-be exemplified in this maimer: 

UtX = ^,Y, =^,Y2^=AAS; . (6) 

then4'=f(*,AAS) . ' 

where 0 = X- for some i as shown in Figure 25. 
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$^ (t> is a pitch angle data sample at t 
^ « 

(t) is' the valu6 of the criterion function at t 
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/ ' b. Threshold Error Measures. Associated with the described mean reference function Xj is thp 
standard deviation function X^i. A representation of this measure appears in Figure 26. A ttueshqld enor 
function can be constructed from X<rj, as follows: . ... 

Ej, = Oif lE^KXa 

(7) 

.E„=l if |E^|>Ka. 

and - . 

^<,i = Eai(K;X;Y,,Y,,...,Y„) . ' ■ . ' (8). 

The parameter K specifies the number of standard deviations and i distinguishes among the E^. - 

c. Differentia! Difference Error Measures. A differential jor difference equation representation of the 
high performance maneuvef flights can also be used as a maneuver critenon. Let the correjpondmg cntenon 
be represented as follows ' ^ , " 

• DX = g.(X;Yi,Y„...,Y^) " ^ (9) 

where DX indicates mean change (mfean over excellent category flights) m variable X oyer .time period At. 
Note that X can be included in the ar^ment. Subscript i is used to identify a particirfar function. The 
corresponding error Ts 

In- - . ^ • 

«x = N'2DX-g.=ex(X;Y„Y,,..,,Y^)- ^ ' ' (10) 

■i=l " - ■ . • ■ . . 

and the total measure over all variables is: 

Ejjj^ Vi^i = X,,X,,...,X^ '■ ('11)' 

Aj = ^j(^i>^> - >^'Yj,Y2,. '.,Y;) ' (12)' 

where j indicates a particular set of difference functions. 

d. Discrete' Task Error Measures. In reference to Figures 22/23, and i24, it seen that discrete task 
errors are measured by observing the value of a continuous variable when the task is accomplished 
successfully or observing the preceding and succeeding tarics. Consider the value of variable (V) when 
discrete task X is completed; e.g., when X=l test value of V. 

An event (discrete task) error can be formulated as 

EE=0ifV^^<V(X)<V^2 ' ' ' 03) 

Eg = 1 if. V(X) > °' < V^.^ 
Note that ' '. 

Eg = Eg(X;V^^,V^2;V) ; ' (14) 
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where X idcrftifies the event that triggers th^ measurement and Vt2,Vti Identifies the tolerance limits of 
the tested variable and V is the tested variable. Also Vji^ Vt2 can in^cate. bounding discrete events for 
event sequence tests. Enor function Ee is true whenever the tested variable value is not within the 
prescribed toTerancc.Tigure 27 shows ari example of the usage of Ee . 

^ An enor measure that indicates the amount of out-of-tolerance is \ 
^Evj = Maxj(V-^V^,)-V,V-<V,^fWT>} • * ' (15) 

Notation for this type of prinury enor is 

Figure 28 shows an example of the usage of Ey. In equation 16, Vji represents the lower tolerance lim|t, 
Vt2 the upper tolerance limit. " 

e. Sample Error Measures, In addition' to enor measures, it is desirable to sample the value of 
specified variables at specific tinies during the maneuver segment. For example, a maximum or minimum 
may be required. Notation for tiiis measure is w« 

Es.(Y;X) ' ^/ (17) 

V 

where E5 indicates a sampled measure, j the number of the measure, X'the variable to be sampled, atid Y 
the condition for sampling. For example, if we want to measure heading ^ when roll angle reaches zero, the 
notation would be 

Esj(* = 0;^). " ^ . • (18) 

Also, ff we want to measure maximum pitch angje the notation would be 

f. Miscellaneous Error Measures, The trigger variable identified as X in Equations 13, 14, and 16 can 
use several conventions, each of which indicates when a test-is to be performedfl One notation used js a logic 
notation, e.g., x - 1, 0 = 0. Another notation is 8^^ and 0mj where ^oi refen to the ith time 6 is zero during 
the maneuver and 6^^ refers to the jth time 0 reaches a naximum or minimum during the maneuver. Thus, 
if a measurement is desired on say,0 when 0 reaches the first extremum (say max), the notation could be 

Eg.(o^^,-2^+2^^). - ■■ . 09) 

This enor measure notation specifies that when 9 reaches its first extreme value, test ^ for a zero value ±2^. 

Previous enor measures were defined with sin^e index systems. This might be insufficient to take 
care of all cases 50 a d^ouble index system may be usefiil, to wit, Eykj and EejJ. 



Operations pn Error Measures . . . 

TKc described enor measures are the raw data from which candidate proficiehcy measures^are \o Ke 
derived when judidoudy ch(»en (Operations are performed on such data. These operations are described in 
the following paragraphs. ^ ^ \ . ' • 

a. Linear Sums, One type operation is, the sum of weighted measures. This can be niathematically 
expressed^: , ' ^ 



(| <1>| - 5^ 200. 230, AS) 
WHEN I o| «5^TESTAS 
Eg-OJF-200 <AS <230 OR 

Eg-1JF AS ^230-' 
AS <200 




200 



230 



Figure 27. Error type E£. 

E^ ([<^ I = 180''. h^^-25. h) 
WHEN I ^ I 180"*, TEST:$LT (h) 
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Figure 28. Eritor Type Ey . 
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This function can be represented as. , ^ 

p^CE,,...,Ej^) : ' " : • (21) 

where Ek can be an enoi measure or a sum of error measures. The subscript K indicates a weighted sum is 
used. Equation 20 is in gciieralized form and can be made specific thusly: r ' ^ • 



K 



\ 



Pk = K^2 W.E^. \ (22) 
i=l 

The weights may be computed in' any of several alternative ways. The optimal way is unknown 
and depends to a great extent on the nature of the measures to be weighted and sumrned. In sonie cases, 
weighting may not be appropriate at all, in which event the Wj would be set to 1. In other cases weights 
might be derived by regression analyses using some know^Hndex of skill (independently derived measure) 
employed as a basis for overall performance discrimination. In still other cases, it may be desired to study 
the weighting of mdmdual measures m a way which woducej^ weighted summary measure satisfying some 
fundamental measurement theory or cpncept. An ^umpl^f the latter would be the concept of njinimum 
performance variance in high^ skilled performers\in/Wiich case an experimental weighting technique 
which achieve^ variance-minimiiation would be explorwL 

In view of the many weighting alternatives that exct^dto one method was singled out for application 
to all measures in all segments of the v^dus maneuvers under study. Instead, a capability was developed to 
permit study of any technique "by allowing the researcher to specify the desired method for each^case. A 
"minimum variance" method, for instance, was developed mathematically (Appendix A) and implemented 
in the software research system, however its use is optional, allowing it to be studied comparatively with 
other altcmatiye methods. 

b. DivergenceJConvergence Weighting. An error can be weighted based*Dn the sign of its derivative. 
Th^s, if * . ' , , 

(24)' 

M)i > iEKa+i)i.w^=o,. . • 

where the" subscript j denotes specific t^me samples. This can be expressed compactly as: 

c. Maximum Term, Another method of combining measures is by the selection pf the largest error 
term • • * ' 

' PMax,^OVi,E.) = Max)w,E. W^E^ [ (2.6) 

• ^ * 

where Wj is a weighting factor on the error. . ' • ^ 

Weights for each error term are such that the expected value of each weighted factor for the excellent 
(IP) performance category are equd. These weights once determined, are used to form the candidate 
' proficiency measure Pm^xk- 

d. Discrete Factor Parameterization of Continuous Measures, T^e discrete factor parameterization of 
continuous error measures is used to provide differential weighting of continuous task errors as a function 
of the existence of discrete tasks (a secondary task). For example, notation for the operation is 



Ppj = Ppj(Xi;Ei) ^ (27) 

Wi is the weighting factor value of Ej. When discrete factor is true, = K, and when is false, W, = K[, 
Optimum values for Ki,K| have no^ been established. * * - . 

e. Vector Operation. The final ojJeration provides individual enor components in vector form. This 
can be represented as **vector addition" ^ . . 

^ P- = e-^2EE. + e^2E,. + e-^2E^: + e- SE^.VT^ E^^ , . - . . 

1 1 1 J . i ^ ^ 

where ce, cq, cm , ey and eo are orthogonal unit vectors. 

The notation^ expression is * 

P-.(E,,.:.,eJ : ' ' (29) 

A generalized "vector addition" can be used to process jneasures of similar types of tasks rather than types 
of measures as shown above. 



VnL SEGMENT AND MANEUVER MEASURE FORMULAS 

This Section documents the specific formulas for candidate measures for each of the five maneuvers. 
The measure types (e.g., EM, EE, PK) and their parameters were de^ribed in detail in the preceding. 
Section. Therefore, the definitjions 6f parameters for each measure type, as provided in the following tables, 
completely define the measure formulas. i 

Table 6 presents the basic notation used throughout the subsequent measure specification Tablfs 7 
through 21. 



IX. OVERVIEW OF SOFTWARE FOft COMPUflNG CANDIDATE MEASURES 

Software was developed for computing user-specified measures from recorded operator performance 
data, and producing hard copies of results for analysis and subsequent validation tests. The software is 
designed to compute any measures of the general types discussed in Section VIL Spedflc measure fcrmulae, 
as presented in Section VIII to document candidate T-37 measures, are represented by their parameters, 
specifiable at run -time by the user. Therefore, the software has utility for measurement studies additional 
to the T*37 problem addressed herein. t ' ' ^ 

The software, implemented on a Sigfna 5 computer,' performs the following tasks: 

(1) Smooths data to remove noise and/or introduce special filtering in accordance with user's 
specifications. ' * . ' 

(2) Produces print Quts and plots of raw and smoothed data at sampling rates specifiable by the user. 

(3) Automatically segments maneuvers in accordance with the task segmentation results documented 
-in Section III. 

(4) Computes criterion (reference) functions from sets of maneuver performances identified by the 
user (e.g^ instructpr pilot performances or those of skilled student pilots). Dependent an.d independent 
variables are specified at run time by the user. 



'The Sigma 5 is part of a Simulation and Training Advanced Research System (STARS) in the Advanced Systems 
Division, AFHRL,Wright-PattcrsonAFB, Ohio. , * 
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Table 6. Symb9b 



Symbol 


Computtr "Varlsbld 




X 




Boolean Variable 


-1 

Y 

/V 


• 


NOT Operation on 
Boolean Variable 


v 
X 


- 


Menio^ wperaiion wnere ii 
A^^loJ — 1 ,-ana y^yi) i v.tQ 

^hen X (t) = 0 t< to 






X<t) = 1 t>to 




HEADG 


Heading Aircraft 




RU^^WAY . 


Runway Heading 




ROLL 


* Roll Angle 






Roll Angle Required for Turn . 
Coordination (see Appendix II] 


e 


PITCH 


Pitch Angle 




PITCHL 


fitch Angle for Level 'Flight 


h 


ALT ■ 


Altitude 




ARATE 


Altitude Rate (feet/minute) ' 


AS 


AIRSPD 


' Airspeed 
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ENGINE 


Engine (RPM) 
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NACCEL 


Normal Acceleration ^ 
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' Longitudinal Stick -Force 
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JLateral Stick Fqrce 
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Lateral Stick Positign 


6p=Long 


• L0SP| 


' Lon^tudinal Stick Position 
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.PRATE • 


fitchingRate 
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QRATE 


• Rolling Rate 
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YRATE, 


Yaw Rate . 
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LACCEL 


Lateral Acceleration 
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Table?, Caqdida'te Segment Measures for Splits 
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Tables. Special Terms Used* in S^ment 
s Measures Sp&ification for SpHt S 
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X, 

X3 
X4 

Xs 
X« 
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Table 9. Segment and MaiK\tver Summfuy 
Measures for Split s 
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Table 11. Special. Tenns Uaed in Segment 
> • Measures Spedfication for Approach 
. - aodl^nding " 



"•"•rni ' Otflnltlon 

^ i 

^1 Tape Maik 

X4 101 = 5° 

^ • • ■ m 1^1=15° 

X7 ' . . . 10! < 5° 

X, . I'^'A -'^'1=90° 

X, 0 101 = 5 

Xii . T = 45 

X,2 . .- h = 0 
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Table IZ S^ment and Maneuver Summary 
Measures for Approach and Landing 

Snmint ■ Sumirary -Mtsun 

I. Enlry PK(EE,EV,E<;) 

PMAX(EE,EV, E<7) • 
(45° Turn) " PK(EE,EM) . 

PMAX(EE,EM) 
II. Initial PK(EE,EV) 

PMAX(EE,EV) 
ni. RtchOut PK(EE,EM) 

PMAX(EE,EM) 

IV, Downwind TK(EE) 

PMAX(EE) 
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(End Final Turn) PK (£E, EM) 
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PMAX(EE,EM,Bd) 
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' PMAX(EE,EM) '• 
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PMAX(EE,EM,ES)' 
Total Maneuver PK (PK). 
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Table 14. Spedal Tenns UscB in 
, ^ ^ S^entMeatsutps Specification 

; for Barrd RoD 

, Ttrm , Dtflnitlon 

r ^ 

X, (101 = 5)n(^FO) 

• Xa .0^i = 5)nXj 

X4. 1^1 = 90 

► Xj 4> = m 

■ X« |«| = 90 - 

X7 ^ = 0 • _ 

■ ^o- .//(Tx.) ■ 

Gi . Uppisr Limit on G's 

(to be determined) 



Table 15. Segment and Maneuver 
Summary; Measures fox.Banel Roll 

S*9m«nt Summary Mujur* 

I. Initial PK (EE) 

II. Entry PK(EE,EV) 
ni-VI, Quadrants .PK (EM, Ea) 

1,2, 3, and 4 " PMAX (Eivl, Ea) 

Pd(EM) 

in/^adrantl . PK(EE) 

IV. ^atrant2 PK(EV) 

V. Quadrants " PK(EV) 

VI. Quadrant 4 PK (EE, EV) ' 
Total Maneuver PK (PK) 

PK<PMAX) 
PK(Pd) 
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Table IZ Spedil Terms Uaed in Segment 
Measures Specification for Cloverleaf 



Ttrm -DtflnH lon 

X, • fl = 0 

X3 X^>40) n (|^p>5) 

X5 't<tew. 

ASo AS(tx,)- 
ho h(tx,)- 

; ^l''o-^^0(SGN(.|,(tx/)-./'ioJ) 
^^'3 , 'i/zj+lSO 



Table 18. Segment and Maneuver 
Summary Measure* for Cloverieaf 



S«fnitnt 

I. Initial 
II. Pitch Up 

ni. Turn , 

IV, faS, Through 

(End Pull Through) 
Total Maneuver 



Summary MMtur* 



PK(EE) 
PMAX(E'fi) 
PK(EE,EM,Ea) 
PMAX'(EE, EM, Ea) , 
Pd(EM) 

PKCEM.'Ea.Efi.EV) 

PMAX(EM, Ea, EE.EV) 

Pd(EM) ' 

PK(EE, EV, EM.Eo) 

PMAX(EE,EV,EM,Eo) 

Pd(EM) 

PK (EE, EV) 

PMAX(EE,EV) 

PK(PK) . " 

PMAX(PMAX) 

Pd(Pd) 
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Table 20. Spccud Terms Used in Scgpient 
"Measuits Specification tot Lazy 8 



T«rm 




Dtflnltlon 




t 


Tape Marie >- 






9 = 10 






l^->/'ol=45 






' 1"/' -j['oT = 90 


Xs 


1 








0 = 0 


X, 




J^-"/'o'l = 45 


x. 




l\/'-l^o'l = 90 


X, 




l"/'-V'o'l = 135 


Xio 




0 = 0 . 


ho 


/ 

. / 


h (tape marie) 


ASo 


AS(Tx,) 


*o 






M> 








^(Tx2) + 180 


• 





Table 21] Segment and Maneuver Summary 

Measures for Lazy 8 

« * 

Stgmint ^ SuMnury Mtasur* 



I. Initial 

II. Quadrant 1 

ni. Quadrant 2 

IV. Quadrant 3 ^ 

V. Quadrant 4 
VI. Quadrant 5 
Vn. Quadrant 6 

VIII. Quadrant 7 

IX. Quadrant 8 
(End) 

Total Maneuver 



EE 

PK(Efe,EM,ES) 
PMAX(EE,EM,ES) - 

' PK(EE,EM,ES) 
PMAX(EE,feM,ES)' 
PK(EE,EM,EV) ' 
PMAX(EE,EM, EV) 
PK(EE,EM) 
PMAX(EE,EM) 
PK(EE,ai,EV,ES) 
PMAX(EE,EM^EV',ES) 
PK(EE,EM) 

, fMAX(EE,EM) 
PK(EE,EM) . ' 
PMAX(EE, EM) 
PK(EE,.EM) 

' PMAX(EE,EM) 
PK(EE,EV) 
PMAX(EE,EV) 
PK(EE,PK) • , 
PMAX(EE,PK) 
•PMAX(PMAX) ■ / 
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(5) Computes, for each maneuver segment, measures specified by the user of the following forms, 

(6) Computes, for each maneuver segment and each maneuver, summ'ary measures of the forms P] 
'Pd» Pmax» Pp, with weighting factors specified by the user as appropriate. 

(7) Prints results of all maneuver segmentation, criterion functions, and measure computations. 

(8) Performs validation tests^ on comf)utcd candidate measures and prints results. 



X SUMMARY AND CONCLUDING REMARKS 

The purpose of this study was to develop candidate T-37 pflot performance measure? for 5 contact 
ifianeuvers and software techniques for^omputing and testing them using recorded flight oata. The results 
of the study include a CQmpreh(|li|^^of candidate e,rror measures developed on the basis of apparent 
content validity; sever^ alternative methods of combining measures to form overall performance 
assessments as required; and a flexible ^ftware system for computing and testing fqr validity the measures 
herein developed (for the T-37) and any.other uset-specified measures of the general types incorporated. 

The results also include a uniquernfiasarement-oriented method of operator task analysis and 
segmentation, and its application topWmaneuvei? tau^t in the Air Force UPT program. The method 
includes identification of two Xy^yt€{ function segments (locus .and sequence) within a given control task, 
whereiit the set of dominant iric^uremcnt variables is consistent. This identifies portions of each task in 
which the operator's primary control function involves consistent njeasurable variables, and suggests the 
types of measures (continuous and discrete) applicable to assessing the operator's control performance. The 
method also includes identification of task segments, wherein the relationship among dominant 
measurement variables is consistent. This, identifies portions of each function segment. in which the 
operator's primary controffiinctions themselves remain consistent, and suggests the specific natu/e of the 
continuous or discrete roeasure^ applicable to performance assessment within the respective task segments. 

Based on the applicatio;i of this analysis technique to 5 UPT maneuvers, several types of measures 
were identified and defined algorithrhically. Collectively, they support performance assessment in all 
maneuver segments. Then, specific measure formulae were derived for each segment. Fipally, software was 
developed and implemented for computing user-specified measuies and validation test results using 
recorded T-37 data. ^ ^ ^ 

The approach employed is one of two that have been identified 'for the general job of developing and 
testing candidate measures for operator performance tasks. In it, the researcher identifies th^ specific 
measures to be tested, assuring their content validity, and then the measures are computed and enipirically 
tested for criterion.-related validity. (The alternative approach (see Connelly et'al., 1974) wliich was pursued 
concurrently involves computer generation of candidate measures from a broad spectrum of measure types, 
execution of criterion related validation tests, and, lastly, researcher analysis of results and assurance of .the 
content validity of derived measures.) The approach was applied successfully to develop candidate 
performance measures for 5 UPT maneuvers in a relatively systematic and thorough way. 

This study was originally intended to extend through the validation phase ot measurement 
development using T-37 student pilot performance data. Unfortunately, non-technical problems prevented 
the data collection, and the study had to be confined to developing candidate measures as reported herein. 
It is recognized that the work performed essentially amounts to developing the tools but never testing 
them. Content validity alone, however carefully assured, is not a substitute for empirically derived 
criterion-related validities. 



These consist of three empirical validation tests designed and documented under a scj>aratc concurrent study 
(Connelly et aL, 1974) and appHed here as p*rt of the overall software system. A brief description of the tests is provided 
inAppchdixC. ^ \ , ■ 
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Despite this kind of "stopping short," the study produced some novel concepts and techniques for 
analyzing perfoijnancc tasks for measurement purposes, and a relatively flexible software system for use in 
continuing measurement research efforts. It also produced a comprehensive set of candidate measures of 
T-3T pilot performance', and a thorough analysis, specifically performed for measurement applications, of 
five representative UPT maneuvers. Hopefully, the .woric will, as a minimum, serve as a guidelme for 
investigation of similar measurement problems, and inspire other efforts for pursuij through and including 
finial validation and evaluation of results. 
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^APPENDIX A: LEAST VARIANCE WEIGHTING . 

OF LINEAR SUMS T • * / 

Independent Measurements - > 

Consider the case of approxiitiating a continuum by a discrete selection of outputs. A single quantity, 
X, is measured by a variety of independent methods that yield unbiased results but each with a different 
accuracy. ^ . . . 

Let the set of measutes of x be Xi, i = 1, 2, . . . , N and' let the corresponding variances be Oi^ 
respectively. S6me methdd of combining the Xi into a single resu)t is required. One method is to add the Xi 
together in a weighted f^on. * • 

WjXi (1) 

i=l 

i 

where the wj are the weighting numbers and X£ i? the expected value of x. Hie ensemble average of xe is 

=f ^i^i = ]^ w^3c. = £ w. ^ ^ (2> 

. i=l . i=r i=l 

where Xi = xt, the true valiie of x. Thi5n, if the ensemble average of Xe is to equal the true value ofx^is 
necessary to constrain Wi sUch that, ^ ' 



t, Wi = l 



' (3) 

i=l . • . . • v 

Since the variance a measure of accuracy, dnd the, "best" variance is desired, "best" c^ be defined 
to mean the smallest vari^ce.^We would like to choose the Wf ip minifra2e*a^ , where 



<^*(xe-v' ' " ■• ^ ■ ' ; , . ' (4) 



jfw,w.(x.-x.)(x.-X) . >■/ 
i=lj=l 



(5) 

i=lj=- - - 



If the Xi are independent, . , ■ ' ~ 

/ * ' ' ' * t 

- <^='f w,^<'i^ ' • . ' . ' ^ ' (6),' 

1=1 - • ^ ' 

Now, the Wi must be .chosen to miniinize , )^hile the sum of the wj is constrained to equal unity. 

An auxiliary problem with no constraints is'^^n^ercd, such that if it is ^solved^ then our original 
problem is solve£. Coi^der the pioblenf of choosing the wi and X that 



^where X is the La & ra ngan ifmltiplier. A necessary coiwfition for (he minimization of S is that each of its 
partial derivative wiSi respect to wj and X be equal-to zero! 



2S=^W5-l=0.. 



(8) 



If the constraint is subsQtuted IntoThe equation for S, it can be seen that only o? remains to be reduced. 
Thus, the solution of the auxiliary problem without constraints is equivalent to the solutioffof tlie original 
pr oblem. . ^ ^ - 

% M ^ ^q .i=l ^V- - 
where 

— = o = 2a'w + X, ■ 00) 

, ^ 1 . . 

w=-X(-T~) q=l,2....,N , ' 00 

^ . ^ X 

Summing on q,' setting the sum equal to 1 , and solving for (- -) 

/ 

X _ 1 " ' » ■ 



2 /N \ (12) 



Thea 

w = ( 



■w=(_L.)- / ■ y • • . , (13) 

Thereforft the best estimate of x is: 



E ^ r ^ ■ 



After cpmbining equations (6) and (13), the'Variarice is: » * . 

a» = _i : 

N J_/ ^1, ' • 05) 

' X (a ^ , ; ' ' - 

q=l 1 < - . . . . ' 



pependenTVanabl^ ~ 

When measurenicnts arc not independent, the simplification from equation (5) to equation (6) cannot 
bemaideand, • - 



) w.w. v.. 



i=i j=i 

where the vy are the cov^ances: 



The new problem has the same constraint expressed in equation (3) 
N ' . 

i=l 



afid the wj must be chosen to rninimize . Similar "to the above equation (7) 



N N 
^ = / > W.W.V.. + X 
- 1=1 j=l 



i=l 



As above 



(16) 



(17) 



-(3) 



(18) 



N 

,1. 



(19) 



sinde Vi, = Vqj, it 




q=J,2„... N 



(20). 



(21). 



Equations (21) are^N Jin^ir^qu|itions with N unknowns. A solution for the f , exists if the determinant of 
v^ is not zero. I^lihrelenients of the inverse matrix be Cjj.-Then 



Ev.'c . = 5.. 
■ iq qi « 
q=l ^ 



4 



X 



(22) 



Multiply equation (21) by Cqj and suni on q 



N 



. w,=(-2)f c . 



2 qs:!' 
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(23)- 



Sumnung on j and solving for (- ^ 



1 



2' /N 



(24) 



Therefore: 



w. 



, q=l i=l 

, Substituting into equation (16)^ the variance of the best Mtimate becomes: 

1 N N N N 

E E I n d .c X _ 



Using equation (22) 



qS 



1 



Using matdx npistion 



lct4ij = l,i'=l,2,...,N 
Equation (i)1)econjes = w!^Vw 



Equation (3)jKCGpes 'u^w.= 1 . 
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(25) 



(26) 



(27) 



(28) 



(29) 

•(ao) 



TJie problem is to choose w to ^nmimize a* while satisfying equation (30). Equation (18) becomes 

S = w'''Vw + X ju'^^w-lj" (31) 



^uTw-l=0 . ' - - (32) 



5S 2wTvS5w + Xu^ 5w - (33) 

where the first tcnn in equation (33) resuhs from the fact that a scalar is equal to its own trantpoce, and 
that V = V*^^ therefore: 

5S= hw'^V + Xu'^J 5w ' ^34) 

5S = o for an 5w 

wT=(_^u'^V^ (35) 
The transpose of equation (30) is taken and equation (35) is multiplied from the right by u, then solved for 

(-V2) / . ; ^ ' 

(_V2)=^ — i ' • ^ \ , ; . (36) 

u'^xr^u 

and equation (35) becomes 

w^ = ^ * . . 

uV^u. ^ • 

then<^ ^ - 

J u'^vW(V^)Tu 

'(u^V-iu)^ ~ 
but(V^)*'" therefore: 



3 ' ■> 



The correspondence with indexed notation is complete with J_ / 

^ N N ■ . . " ^ : 

i=l q=l ■ 

and ■ 

N .. ., '■• : ■ . 

q=l • > ' . 
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APPENDIX B: MEASUREMEhrt' OF TURN COORDINATION 

Attame that the prctent turn provides a constant rate of rotation about a point distance r from the 
lircnft. MearaAnent of the rotation rate w can be provided by the rate of diange of heading, thus. 

This turn rate provides an acccferation component assunt^ to be perpendicular to g (i,e^ alevel turn) 

where V is aircraft velocity, wHdi will be measured by ainpeed (zero air velocity with respect to ground is 
assume<l). Thus the proper roll angje for a coordinated turn is 



, I viK| 



«^ =TAN-' 

This roll angle is to be used as a reference hi measuring the degree of coordination of a turn. 



\ 
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^ APFENDDCC: VALIDATION TESTS^ 



For the vast imgority of pcrfonnance taski, there is no tin^e necessary and fufiSdent test that can be 
appHed to candidate measures to assess their validity. Measures ^ch appear to hive content validity often 
fail to reHably disciimmate even between novice and higWy experienced performers. Measures which appear 
to have concurrent validity nuy or may not satisfy oAcr validation criteria, depending on the reHabOity 
arvl sensitivity of tlie metric used as a basis of comparison. 

The approach hi this study was to develop three enqnrically-based validation tests to be appHed by 
the measurement processor. Collectively, the tests are used to determine tlie HkeHhqpd that eadx candidate 
measure is vaBd . Final analysis and assurance of the measure's content validity is performed by the user of 
the processoi^, based on evidence accrued by it and printed out for his consideration. 

The first test assesses the measure's potential contribution to discriminating between performances at 
opposite ends of the skill continuum. The data employed for this test are selected by the user. For the T-37 
pilot performance tasks that were to have been addressed here, the following two types of data would have 
been investigated: 

(1) FH^ts flown by instructor pilots to demonstrate their best performances and simulated novice 
performances of each maneuver. 

(2) Flints flown by students at the neophyte stage and at the successful completion of training. 

The techniques implemented to apply this first test hidude (a) comparison of residues from regression 
analyse^; and (b) the rank sum statistic. 

The second test assesses the measure's functional relationships with variables such as number of trials 
and time in training. A measure yAskii demonstrates that learning has occurred from neophyte to 
experienced levels of performance would posiess a higher Hkelihood of validity than one which consistently 
does not, for example. Again, the data to be employed for this test are specifiable by the user. For Ae T-37 
pflot tasks, the following data would have been exjteiimented with: 

(1) Time in traming 

(2) Number of practice sorties on the maneuver 

(3) Number of practice tri^ pn the maneuv^ 

The technique used to ap|dy this test consists of developing an4 analyzmg a multi-variable regression 
function. (An ahemative tecfanqiue based on the use of Maricov kamhig modds wu conceived, but due .to 
lade of data, has not yet been djcvdoped to the pohit of imidementation.) 

The third test assesites the n)easure*s functional reUtionships with subjectively derived ordinal scale 
measures of performance. Measures >^Wch tend to refaforce the subject^ve ordering of performances are 
. considered more likely to be vaHd than those which cpnsistcnfly M to do to/tbt dataemi^oycd for this 
test, as with the other tests, arc specified by the iflfer. For the T-37 tasks, inirtmctor pflot^flngs^uld have 
bc«i hivestigated for use. The tedmique for applying the test is to dcvdop and anal^zemdti^Va^^ 
r^ssion functions, sis hi the second test described -to the preceding paragrai^ 




